{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "be881a6d",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "source": [
    "# Retrieval Augmented Generation (RAG) with OpenAI and Qdrant\n",
    "\n",
    "In the ever-evolving landscape of AI, the consistency and reliability of Large Language Models (LLMs) remain a challenge. While these models can understand statistical relationships between words, they often fail to provide accurate factual responses. Because their internal knowledge may not be accurate, outputs can range from spot-on to nonsensical. Retrieval Augmented Generation (RAG) is a framework designed to bolster the accuracy of LLMs by grounding them in external knowledge bases. In this example, we'll demonstrate a streamlined  implementation of the RAG pipeline using only Qdrant and OpenAI SDKs. By harnessing Flag embedding's power, we can bypass additional frameworks' overhead. \n",
    "    \n",
    "This example assumes you understand the architecture necessary to carry out RAG. If this is new to you, please look at some introductory readings:\n",
    "* [Retrieval-Augmented Generation: To add knowledge](https://eugeneyan.com/writing/llm-patterns/#retrieval-augmented-generation-to-add-knowledge)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb044259",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "Let's start setting up all the pieces to implement the RAG pipeline. We will only use Qdrant and OpenAI SDKs, without any third-party libraries. \n",
    "\n",
    "### Preparing the environment\n",
    "\n",
    "We need just a few dependencies to implement the whole application, so let's start with installing the dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4ce9f81b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:32.977456Z",
     "start_time": "2023-09-27T10:06:30.203757Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: qdrant-client in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (1.5.4)\n",
      "Requirement already satisfied: fastembed in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (0.0.4)\n",
      "Requirement already satisfied: openai in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (0.28.0)\n",
      "Requirement already satisfied: grpcio>=1.41.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (1.58.0)\n",
      "Requirement already satisfied: grpcio-tools>=1.41.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (1.58.0)\n",
      "Requirement already satisfied: httpx[http2]>=0.14.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (0.25.0)\n",
      "Requirement already satisfied: numpy>=1.21 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (1.26.0)\n",
      "Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (2.8.2)\n",
      "Requirement already satisfied: pydantic>=1.10.8 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (1.10.12)\n",
      "Requirement already satisfied: urllib3<2.0.0,>=1.26.14 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from qdrant-client) (1.26.16)\n",
      "Requirement already satisfied: onnxruntime<2.0.0,>=1.15.1 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from fastembed) (1.16.0)\n",
      "Requirement already satisfied: requests<3.0.0,>=2.31.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from fastembed) (2.31.0)\n",
      "Requirement already satisfied: tokenizers<0.14.0,>=0.13.3 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from fastembed) (0.13.3)\n",
      "Requirement already satisfied: tqdm<5.0.0,>=4.65.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from fastembed) (4.66.1)\n",
      "Requirement already satisfied: aiohttp in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from openai) (3.8.5)\n",
      "Requirement already satisfied: protobuf<5.0dev,>=4.21.6 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from grpcio-tools>=1.41.0->qdrant-client) (4.24.3)\n",
      "Requirement already satisfied: setuptools in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from grpcio-tools>=1.41.0->qdrant-client) (68.2.2)\n",
      "Requirement already satisfied: certifi in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpx[http2]>=0.14.0->qdrant-client) (2023.7.22)\n",
      "Requirement already satisfied: httpcore<0.19.0,>=0.18.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpx[http2]>=0.14.0->qdrant-client) (0.18.0)\n",
      "Requirement already satisfied: idna in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpx[http2]>=0.14.0->qdrant-client) (3.4)\n",
      "Requirement already satisfied: sniffio in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpx[http2]>=0.14.0->qdrant-client) (1.3.0)\n",
      "Requirement already satisfied: h2<5,>=3 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpx[http2]>=0.14.0->qdrant-client) (4.1.0)\n",
      "Requirement already satisfied: coloredlogs in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from onnxruntime<2.0.0,>=1.15.1->fastembed) (15.0.1)\n",
      "Requirement already satisfied: flatbuffers in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from onnxruntime<2.0.0,>=1.15.1->fastembed) (23.5.26)\n",
      "Requirement already satisfied: packaging in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from onnxruntime<2.0.0,>=1.15.1->fastembed) (23.1)\n",
      "Requirement already satisfied: sympy in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from onnxruntime<2.0.0,>=1.15.1->fastembed) (1.12)\n",
      "Requirement already satisfied: typing-extensions>=4.2.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from pydantic>=1.10.8->qdrant-client) (4.8.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from requests<3.0.0,>=2.31.0->fastembed) (3.2.0)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (23.1.0)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (4.0.3)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)\n",
      "Requirement already satisfied: hyperframe<7,>=6.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from h2<5,>=3->httpx[http2]>=0.14.0->qdrant-client) (6.0.1)\n",
      "Requirement already satisfied: hpack<5,>=4.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from h2<5,>=3->httpx[http2]>=0.14.0->qdrant-client) (4.0.0)\n",
      "Requirement already satisfied: anyio<5.0,>=3.0 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpcore<0.19.0,>=0.18.0->httpx[http2]>=0.14.0->qdrant-client) (4.0.0)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from httpcore<0.19.0,>=0.18.0->httpx[http2]>=0.14.0->qdrant-client) (0.14.0)\n",
      "Requirement already satisfied: humanfriendly>=9.1 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from coloredlogs->onnxruntime<2.0.0,>=1.15.1->fastembed) (10.0)\n",
      "Requirement already satisfied: mpmath>=0.19 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from sympy->onnxruntime<2.0.0,>=1.15.1->fastembed) (1.3.0)\n",
      "Requirement already satisfied: exceptiongroup>=1.0.2 in /home/user/.cache/pypoetry/virtualenvs/qdrant-examples-C1NRJP3C-py3.10/lib/python3.10/site-packages (from anyio<5.0,>=3.0->httpcore<0.19.0,>=0.18.0->httpx[http2]>=0.14.0->qdrant-client) (1.1.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install qdrant-client fastembed openai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aae4382a",
   "metadata": {},
   "source": [
    "[Qdrant](https://qdrant.tech) will act as a knowledge base providing the context information for the prompts we'll be sending to the LLM. There are various ways of running Qdrant, but we'll simply use the Docker container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e8f4456c",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:34.283299Z",
     "start_time": "2023-09-27T10:06:32.980517Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "066db7a31c0261cbbb221eec95b9ce214ae306b6601afbdb03cefa068a9cf9ed\n"
     ]
    }
   ],
   "source": [
    "!docker run -p \"6333:6333\" -p \"6334:6334\" --name \"rag-openai-qdrant\" --rm -d qdrant/qdrant:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a74c7a21",
   "metadata": {},
   "source": [
    "### Creating the collection\n",
    "\n",
    "Qdrant [collection](https://qdrant.tech/documentation/concepts/collections/) is the basic unit of organizing your data. Each collection is a named set of points (vectors with a payload) among which you can search. After connecting to our running Qdrant container, we can check whether we already have some collections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2dd8966b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.242783Z",
     "start_time": "2023-09-27T10:06:34.289290Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CollectionsResponse(collections=[])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import qdrant_client\n",
    "\n",
    "client = qdrant_client.QdrantClient(\"http://localhost:6333\", prefer_grpc=True)\n",
    "client.get_collections()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23f54205",
   "metadata": {},
   "source": [
    "### Building the knowledge base\n",
    "\n",
    "Qdrant will use vector embeddings of our facts to enrich the original prompt with some context. Thus, we need to store the vector embeddings and the texts used to generate them. All our facts will have a JSON payload with a single attribute and look as follows:\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"document\": \"Binary Quantization is a method of reducing the memory usage even up to 40 times!\"\n",
    "}\n",
    "```\n",
    "\n",
    "This structure is required by [FastEmbed](https://qdrant.github.io/fastembed/), a library that simplifies managing the vectors, as you don't have to calculate them on your own. It's also possible to use an existing collection, However, all the code snippets will assume this data structure. Adjust your examples to work with a different schema.\n",
    "\n",
    "FastEmbed will automatically create the collection if it doesn't exist. Knowing that we are set to add our documents to a collection, which we'll call `knowledge-base`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "43154775",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.692231Z",
     "start_time": "2023-09-27T10:06:36.245915Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['bd4a7b79a81947f2a5e94b896a468ce1',\n",
       " '8611b165664b4f908710cac0de0330a8',\n",
       " 'b77c4297ea2d4d23a29a5a79feceb250',\n",
       " '26c07c7971444365b4486e4a6925aeed',\n",
       " '539944a8711047b19642c8238231d64a',\n",
       " '3ed7cebd4b354f54b50ee817eebb23e9',\n",
       " 'fb2d523bd50c4563a7d36de3cd2962ca',\n",
       " 'ab94cc63ba53460fbad003a5e2b69e43']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.add(\n",
    "    collection_name=\"knowledge-base\",\n",
    "    documents=[\n",
    "        \"Qdrant is a vector database & vector similarity search engine. It deploys as an API service providing search for the nearest high-dimensional vectors. With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!\",\n",
    "        \"Docker helps developers build, share, and run applications anywhere — without tedious environment configuration or management.\",\n",
    "        \"PyTorch is a machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing.\",\n",
    "        \"MySQL is an open-source relational database management system (RDBMS). A relational database organizes data into one or more data tables in which data may be related to each other; these relations help structure the data. SQL is a language that programmers use to create, modify and extract data from the relational database, as well as control user access to the database.\",\n",
    "        \"NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server. NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.\",\n",
    "        \"FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.\",\n",
    "        \"SentenceTransformers is a Python framework for state-of-the-art sentence, text and image embeddings. You can use this framework to compute sentence / text embeddings for more than 100 languages. These embeddings can then be compared e.g. with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similar, semantic search, or paraphrase mining.\",\n",
    "        \"The cron command-line utility is a job scheduler on Unix-like operating systems. Users who set up and maintain software environments use cron to schedule jobs (commands or shell scripts), also known as cron jobs, to run periodically at fixed times, dates, or intervals.\",\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b36bddd6",
   "metadata": {},
   "source": [
    "## Retrieval Augmented Generation\n",
    "\n",
    "RAG changes the way we interact with Large Language Models. We're converting a knowledge-oriented task, in which the model may create a counterfactual answer, into a language-oriented task. The latter expects the model to extract meaningful information and generate an answer. LLMs, when implemented correctly, are supposed to be carrying out language-oriented tasks.\n",
    "\n",
    "The task starts with the original prompt sent by the user. The same prompt is then vectorized and used as a search query for the most relevant facts. Those facts are combined with the original prompt to build a longer prompt containing more information.\n",
    "\n",
    "But let's start simply by asking our question directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ed31ca63",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.695165Z",
     "start_time": "2023-09-27T10:06:36.695150Z"
    }
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "What tools should I need to use to build a web service using vector embeddings for search?\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7d2d7dd",
   "metadata": {},
   "source": [
    "Using OpenAI API requires providing the API key. Our example demonstrates setting the `OPENAI_API_KEY` using an environmental variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "30e8669e",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.696985Z",
     "start_time": "2023-09-27T10:06:36.696959Z"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Fill the environmental variable with your own OpenAI API key\n",
    "# See: https://platform.openai.com/account/api-keys\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"<< PASS YOUR OWN KEY >>\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bf5c684",
   "metadata": {},
   "source": [
    "Now we can finally call the completion service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d5cdee82",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.700541Z",
     "start_time": "2023-09-27T10:06:36.700518Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To build a web service using vector embeddings for search, you would need several tools. Here are some essential ones:\n",
      "\n",
      "1. Programming Language: Depending on your preference and requirements, you can choose a programming language like Python, Java, or Node.js for building the web service.\n",
      "\n",
      "2. Web Framework: A web framework helps in developing web applications efficiently. Popular choices include Flask (Python) or Spring Boot (Java).\n",
      "\n",
      "3. Embedding Models: You would need vector embedding models to represent your data for search. Some popular models include Word2Vec, GloVe, or BERT, depending on your application domain and requirements.\n",
      "\n",
      "4. Vector Embedding Libraries: To work with vector embeddings effectively, you may need libraries like TensorFlow, Gensim, or PyTorch to load and manipulate the embeddings.\n",
      "\n",
      "5. Database: You would require a database to store and retrieve the information you want to search. Common choices include MySQL, PostgreSQL, or MongoDB.\n",
      "\n",
      "6. Search Engine: You need a tool or library to enable efficient searching using vector embeddings. Tools like Elasticsearch or Apache Solr can be integrated to perform vector similarity searches.\n",
      "\n",
      "7. API Development: To expose your web service functionality, you may need to develop an API. Popular frameworks like Flask or Django (Python) or Express (Node.js) can assist in creating the API endpoints.\n",
      "\n",
      "8. Deployment Environment: You would need a hosting environment to deploy your web service. Options include cloud platforms such as Amazon Web Services (AWS), Google Cloud, or Microsoft Azure, or you can choose a self-hosted solution.\n",
      "\n",
      "Remember to choose the tools that best suit your requirements, taking into account factors like programming language proficiency, scalability, and performance needs.\n"
     ]
    }
   ],
   "source": [
    "import openai\n",
    "\n",
    "completion = openai.ChatCompletion.create(\n",
    "    model=\"gpt-3.5-turbo\", \n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": prompt},\n",
    "    ]\n",
    ")\n",
    "print(completion[\"choices\"][0][\"message\"][\"content\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b420d81d",
   "metadata": {},
   "source": [
    "### Extending the prompt\n",
    "\n",
    "Even though the original answer sounds credible, it didn't answer our question correctly. Instead, it gave us a generic description of an application stack. To improve the results, enriching the original prompt with the descriptions of the tools available seems like one of the possibilities. Let's use a semantic knowledge base to augment the prompt with the descriptions of different technologies!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ce791ba3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.702641Z",
     "start_time": "2023-09-27T10:06:36.702619Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[QueryResponse(id='bd4a7b79-a819-47f2-a5e9-4b896a468ce1', embedding=None, metadata={'document': 'Qdrant is a vector database & vector similarity search engine. It deploys as an API service providing search for the nearest high-dimensional vectors. With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!'}, document='Qdrant is a vector database & vector similarity search engine. It deploys as an API service providing search for the nearest high-dimensional vectors. With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!', score=0.795905590057373),\n",
       " QueryResponse(id='3ed7cebd-4b35-4f54-b50e-e817eebb23e9', embedding=None, metadata={'document': 'FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.'}, document='FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.', score=0.7505609393119812),\n",
       " QueryResponse(id='fb2d523b-d50c-4563-a7d3-6de3cd2962ca', embedding=None, metadata={'document': 'SentenceTransformers is a Python framework for state-of-the-art sentence, text and image embeddings. You can use this framework to compute sentence / text embeddings for more than 100 languages. These embeddings can then be compared e.g. with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similar, semantic search, or paraphrase mining.'}, document='SentenceTransformers is a Python framework for state-of-the-art sentence, text and image embeddings. You can use this framework to compute sentence / text embeddings for more than 100 languages. These embeddings can then be compared e.g. with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similar, semantic search, or paraphrase mining.', score=0.7459007501602173)]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = client.query(\n",
    "    collection_name=\"knowledge-base\",\n",
    "    query_text=prompt,\n",
    "    limit=3,\n",
    ")\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6640067",
   "metadata": {},
   "source": [
    "We used the original prompt to perform a semantic search over the set of tool descriptions. Now we can use these descriptions to augment the prompt and create more context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a16d8549",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Qdrant is a vector database & vector similarity search engine. It deploys as an API service providing search for the nearest high-dimensional vectors. With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!\\nFastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.\\nSentenceTransformers is a Python framework for state-of-the-art sentence, text and image embeddings. You can use this framework to compute sentence / text embeddings for more than 100 languages. These embeddings can then be compared e.g. with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similar, semantic search, or paraphrase mining.'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "context = \"\\n\".join(r.document for r in results)\n",
    "context"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2c04a4e",
   "metadata": {},
   "source": [
    "Finally, let's build a metaprompt, the combination of the assumed role of the LLM, the original question, and the results from our semantic search that will force our LLM to use the provided context. \n",
    "\n",
    "By doing this, we effectively convert the knowledge-oriented task into a language task and hopefully reduce the chances of hallucinations. It also should make the response sound more relevant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1fc9a98b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "You are a software architect. \n",
      "Answer the following question using the provided context. \n",
      "If you can't find the answer, do not pretend you know it, but answer \"I don't know\".\n",
      "\n",
      "Question: What tools should I need to use to build a web service using vector embeddings for search?\n",
      "\n",
      "Context: \n",
      "Qdrant is a vector database & vector similarity search engine. It deploys as an API service providing search for the nearest high-dimensional vectors. With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!\n",
      "FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.\n",
      "SentenceTransformers is a Python framework for state-of-the-art sentence, text and image embeddings. You can use this framework to compute sentence / text embeddings for more than 100 languages. These embeddings can then be compared e.g. with cosine-similarity to find sentences with a similar meaning. This can be useful for semantic textual similar, semantic search, or paraphrase mining.\n",
      "\n",
      "Answer:\n",
      "\n"
     ]
    }
   ],
   "source": [
    "metaprompt = f\"\"\"\n",
    "You are a software architect. \n",
    "Answer the following question using the provided context. \n",
    "If you can't find the answer, do not pretend you know it, but answer \"I don't know\".\n",
    "\n",
    "Question: {prompt.strip()}\n",
    "\n",
    "Context: \n",
    "{context.strip()}\n",
    "\n",
    "Answer:\n",
    "\"\"\"\n",
    "\n",
    "# Look at the full metaprompt\n",
    "print(metaprompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f1a7678",
   "metadata": {},
   "source": [
    "Our current prompt is much longer, and we also used a couple of strategies to make the responses even better:\n",
    "\n",
    "1. The LLM has the role of software architect.\n",
    "2. We provide more context to answer the question.\n",
    "3. If the context contains no meaningful information, the model shouldn't make up an answer.\n",
    "\n",
    "Let's find out if that works as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "709b9f38",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To build a web service using vector embeddings for search, you would need to use Qdrant as the vector database and vector similarity search engine. Additionally, you can use FastAPI as the web framework for building the API. To compute the vector embeddings, you can utilize SentenceTransformers, which is a Python framework for generating sentence/text embeddings. These tools would enable you to create a web service for matching, searching, recommending, and more based on vector embeddings.\n"
     ]
    }
   ],
   "source": [
    "completion = openai.ChatCompletion.create(\n",
    "    model=\"gpt-3.5-turbo\",\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": metaprompt},\n",
    "    ],\n",
    "    timeout=10.0,\n",
    ")\n",
    "print(completion[\"choices\"][0][\"message\"][\"content\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c4120e1-9899-4caa-b974-51d9b3a485be",
   "metadata": {},
   "source": [
    "### Testing out the RAG pipeline\n",
    "\n",
    "By leveraging the semantic context we provided our model is doing a better job answering the question. Let's enclose the RAG as a function, so we can call it more easily for different prompts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "62ed09d1-2c90-4ffc-9f1d-7beb87bab78b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def rag(question: str, n_points: int = 3) -> str:\n",
    "    results = client.query(\n",
    "        collection_name=\"knowledge-base\",\n",
    "        query_text=question,\n",
    "        limit=n_points,\n",
    "    )\n",
    "\n",
    "    context = \"\\n\".join(r.document for r in results)\n",
    "\n",
    "    metaprompt = f\"\"\"\n",
    "    You are a software architect. \n",
    "    Answer the following question using the provided context. \n",
    "    If you can't find the answer, do not pretend you know it, but answer \"I don't know\".\n",
    "    \n",
    "    Question: {question.strip()}\n",
    "    \n",
    "    Context: \n",
    "    {context.strip()}\n",
    "    \n",
    "    Answer:\n",
    "    \"\"\"\n",
    "\n",
    "    completion = openai.ChatCompletion.create(\n",
    "        model=\"gpt-3.5-turbo\",\n",
    "        messages=[\n",
    "            {\"role\": \"user\", \"content\": metaprompt},\n",
    "        ],\n",
    "        timeout=10.0,\n",
    "    )\n",
    "    return completion[\"choices\"][0][\"message\"][\"content\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86fecd76-9a0b-4ad1-9097-b1d292a618ac",
   "metadata": {},
   "source": [
    "Now it's easier to ask a broad range of questions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "aa0fdead-a115-4fcd-88dc-5cc718dc0544",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"The stack for a web API can vary depending on the specific requirements and technologies used, but a common stack for a web API could include the following components:\\n\\n1. FastAPI: As mentioned in the context, FastAPI is a modern web framework for building APIs with Python.\\n\\n2. NGINX: NGINX can be used as a reverse proxy server to handle incoming requests and distribute them to the appropriate FastAPI instances. NGINX is known for its high performance and can help improve the scalability and reliability of the API.\\n\\n3. Docker: Docker can be utilized to containerize the FastAPI application along with any necessary dependencies. This allows for easy deployment and scalability, as Docker containers can be easily replicated and managed.\\n\\nIn addition to these components, the stack may also include other elements such as a database (e.g. PostgreSQL, MySQL) for storing and retrieving data, a cache (e.g. Redis, Memcached) for improving performance, and a message broker (e.g. RabbitMQ, Apache Kafka) for handling asynchronous communication.\\n\\nIt's important to note that the specific stack will depend on the requirements and constraints of the project, and there may be various alternative technologies and configurations that can be used.\""
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rag(\"What can the stack for a web api look like?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7324c127-c140-410a-ab19-87a5babce023",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"I don't know.\""
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rag(\"Where is the nearest grocery store?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fe56730-ed41-42c1-9c33-de3849c60b65",
   "metadata": {},
   "source": [
    "Our model can now:\n",
    "\n",
    "1. Take advantage of the knowledge in our vector datastore.\n",
    "2. Answer, based on the provided context, that it can not provide an answer.\n",
    "\n",
    "We have just shown a useful mechanism to mitigate the risks of hallucinations in Large Language Models."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "968a3ae6",
   "metadata": {},
   "source": [
    "### Cleaning up the environment\n",
    "\n",
    "If you wish to continue playing with the RAG application we created, don't do the code below. However, it's always good to clean up the environment, so nothing is left dangling. We'll show you how to remove the Qdrant container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a0729043",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-09-27T10:06:36.704761Z",
     "start_time": "2023-09-27T10:06:36.704742Z"
    }
   },
   "outputs": [],
   "source": [
    "!docker kill rag-openai-qdrant\n",
    "!docker rm rag-openai-qdrant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97bb6ed9",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
